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FOREWORD - 

The Army Rese^^rch Institute for the Behavioral and Social Sciences has 
been conducting research into improved techniques for the management and 
delivery of technical training for maintenance personnel in the unit 
environment. 

, This research was stimulated by the continuing difficulties that 
operational units experience in developing, retaining and managing sufficient 
numbers of trained mechanics. Work to date has resulted in the development of 
a computer-supported system for tracking and managing mechanics' proficiency 
development. This system -is called. th^ Maintenance Performance System. 

This report by the T>*«ining^and S'l.lniilaH6n fpechnical Area of the Training 
Research Laboratory describes how 1nformat?1on accumulated by the Maintenance 
Performance System can be analyzed to provlae both Immediate performance 
feedback as well as establish long term trend gata oh maintenance 
efficiency. Such information can be of value riot only to unit training 
managers but to personnel and equipment planners as well. 




EDGAR M. JOHNSON 
Technical Director 
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MAINTENANCE TRAINING AND PERFORMANCE: 

A COMPUTER-BASED MANAGEMENT INFORMATION SYSTEM 



EXECUTIVE SUMMARY 



Requirement: 

To improve the management and ielivery of on-the-job technical maintenance 
training. 

Procedure: 

A computer-based maintenance training and performance management 
information system was developed* This system, called the Maintenance 
Performance System (MPS), is designed to monitor the daily technical 
activities of maintenance personnel, identify .maintenance performance 
strengths and deficiencies on an individual and unit level basis, and guide 
training managers to available training resources* 

MPS was installed and made operationaT in a divisional maintenance 
battalion. The need for and use of MPS by training managers at the unit level 
was observed. Also, the practical and potential value of MPS data was 
analyzed. 

Findings: 

MPS succeeded in routinely providing unique and valuable maintenance 
performance and training data; moreover, the cost of supporting the system was 
found to be acceptable to the units. The most significant use of MPS has been 
to guide job assignments. 

Utilization of Findings: 

At the unit level, MPS can help break the pattern of incorrect task 
performance due to lack of supervision and feedback on performance. On a 
larger scale, the MPS longitudinal data base can be used to target Army-wide 
skill deficiencies and fine-tune institutional training curricula, to pinpoint 
areas in which training materials need to be developed or Improved, to 
establish more reliable and comprehensive performance standards, to aid in the 
design of hardware, and to estimate future manning requirements. 
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MAINTENENCE TRAINING PD PEfiFORMANCE : 
A COMPUTER-BASED MANAGEhENT INFORMATION SYSTEM 



Introduction 



The US Army Research Institute (ARI) has designed, developed, and field- 
tested a oon^uter-based management information system which monitors the daily 
technical activities of maintenance personnel. The. primary, goal of this 
system, called the Maintenance Performance System (MPS) , is to improve the 
management and delivery of on-the-job technical training of mechanics. To 
accomplish this, MPS provides unit level supervisors with current informa.tion 
about the experience and performance of maintenance personnel, and guides 
supervisors towar'd selection of appropriate training' materials. The structure 
and operation of MPS is in many ways analogous to existing and planned Amiy- 
wide maintenance reporting systems (such as the Maintenance Control System or 
the- Standard Army Maintenance System). However, MPS differs from these 
reportifig sys^ams in one important respect: MPS is designed to monitor the 
status of personnel, not hardware. 

Does a need exist within the Army for a system that monitors the daily 
technical activity and performance of pe(^ple? Can such a system be 
effectively implemented with minimal draih on existing unit resources? What 
kinds of information can such a system generate and of what value is this 
informationV In tur'n, each of these questions is addressed below and, where 
appropriate, maintenance training and performance- data from an operational' ^MPS 

are reported. ^ 

Why a Training Management Infomation System is Needed 



As a result of the Army's shift to a more decentralized individual 
skill training philosophy, lower organizational units are given greater 
responsibility for the achievement of training goals. In fact, the initial 
and refresher training of individual soldiers in their Military Occupational 
Specialty (MOS) technical tasks is now largely accomplished at. the company or 
squad level (US Government Accounting. Office, 19^1)* Moreover, this training 
is performance oriented and targeted to specific tasks and task standards 
found in each Soldier's Manual. The effectiveness of this training strategy 
is a primary factor in the Arojy's ability to achieve and maintain coitibat 
readiness . 

Several recent reports have suggested that, in practice, soldiers are 
not being adequately trained at the unit level in the performance of their MOS 
technical tasks (see US General Accounting Office, 1981, for a review). This 
lack of adequate training is often reflected in poor performance of technical 
tasks J especially for equipment maintenance specialties. Kern and Hayes (in 
press), for example, reported that 22%-71% of mechanics had one or more serious 
uncorrected errors remaining in the equipment upon completion of their mainten- 
ance task; also, 60% of the mechanics either failed to perform the check"Out 
procedure or did so incorrectly. Buchan and Knutson (1977) reported that the 
frequency of false-removal (removal and replacing a part which is 
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misdiagnosed as faulty) is about 30%. Dressel and Shields (1979) found a 
similar frequency of false-reinoval, and estimated that as much as 30% of the 
total repair cost is attributable to false-removal. Poor performance of 
technical tasks is a phenomenon not only within the Army, but within the other 
services as well (Orlansky and String, 1981). 

When ARI first began its maintenance performance research,. the anticipated 
approach Ws the development of more effective training programs and training 
strategies., However, it quickly became apparent that a more fundamental 
research and development effort had to be- undertaken first, ARI found that, 
at the umt level, there is no official mechanism for recognizing or 
supporting the accomplishment of training. In short, unit level training 
managers have little in the way of resources to pinpoint performance 
deficiencies and to develop unit training programs. Current Army reporting 
systems are biased toward the status of equipment and provide little 
information about the skill and performance of individuals. 

To address this underlying, systemic problem in the management and 
delivery of maintenance training, ARI developed the .Maintenance Performance 
System. MPS is a computer-based maintenance performance management 
information system which provides to training supervisors quantitative 
measures of who needs to be trained, what tasks need to be trained, and how 
training can be accomplished. This information is designed and presenteTso 
that opportunities for training can be easily recognized and taken advantage 
of within the context of a unit's available resources and constraints. 
Examples of MPS information output are given in the third section of this 
report. <« 



Scope of System Operation and System Support Requirements 

MPS is designed for use in a divisional maintenance battalion (although In 
principle it can be employed as well in other technical or combat areas). MPS 
currently tracks ten MOSs and almost all of the equipment serviced by the two 
forward support companies of a divisional maintenance battalion. The MOSs 
include the high density MOSs of 63H (track vehicle mechanics), 63W (wheel 
vehicle mechanics), and 45K (tank turret mechanics). For a detailed 
description of MPS, see Harper (1981). 

Technical MOS supervisors routinely feed information into the system with 
two simple MPS forms. One form is used to record the names of mechanics and 
the work-hours each mechanic contributed to a job; next to each name the 
supervisor checks "GO" or "NO GO" depending on the performance of the 
individual. The second form is used as necessary by supervisors to record 
successful completion by personnel of special training or performance-based 
tests (such as the Army's Skill Qualification Test), Because the use of these 
forms has baen integrated into normal shop routine, data entry for MPS does 
not interfere with regular maintenance activities. Based on observations to ' 
date, supervisors spend no more than ten minutes per week completing these 
forms. 
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Day to day operation of the MPS computer system is accomplished by a 
junior enlisted clerk in grade E2 or E3, The clerk is responsible for 
collecting the MPS input forms, entering the data on these forms into the 
computer, printing computer-generated reports, and distributing the reports,. 
An IBM 5120 micro-computer is used to^store MPS data and to generate the MPS 
reports. The clerk's MPS duties require about 25-30 hours per week. 

The computer programs which allow data entry, data analysis, and report 
generation have been carefully prepared so that very little training is 
required to perform the duties of the MPS clerk. Most of the training can be 
accomplished with a computer-driven lesson in which the trainee "walks- 
through" the operation and use of the computer. 



MPS Training and Performance Data Output 



Mf'S '"•1 been fully operational for about one year at a^ divisional^ 
maintent.. . battalion. The data presented and described below were generated 
^ from the information collected during the first eight months of MPS 

operation. Also included" in these reports are two weeks of data from one 
company of this maintenance battalion that participated in the National 
Training Center exercises. 

Although the following information reflects real field data from an 
operational system, it is important to note that the data are meant to be 
primarily illustrative. The sampT'e of figures and charts below was selected 
because it best represents the scope, uniqueness, and potential utility of 
information provided by MPS — information which is not available from any 
other source. 

Individual Skill History ' 

The Individual Skill History (See Figure 1 for an abbreviated example) 
i'dentifies each mechanic by name and primary MOS, and shows their skill crtdit 
standing listed by specific tasks on equipment items. Tasks specific to each 
equipment are shown on the left column. On the right side, the number of 
skill credits is shown on a scale ranging from one to seven. A star symbol 
(*) indicates that the repairman has gained credit from either on-the-job 
training, other special trainirtg, or from demonstrated acceptable performance 
on the task. 

Skill credits are most often accumulated when a mechanic receives a "dO" 
on either of the two MPS data input forms described above. In Figure 1, 
Mechanic 1, MOS 63H, is shown as having four skill credits (i.e*, training 
experiences) in replacing the engine and transmission on the M60 tank. 
However, only one skill credit was given on Task 5, replacing the 
turbocharger. 

The primary value of the Individual Skill History 1s as an accurate and 
up-to-date skill record and graphic Indicator of a mechanic's standing on all 
required technical tasks, When repair jobs arrive in the shop, supervisors 
use the Individual Skill History for job assignments, For example, the 
mechanic who is inexperienced in replacing turbochargers can be assigned to 
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INDIVIDUAL SKltL HISTORY 



NAME! Mechanic 1 

MILITARY OCCUPATIONAL SPECULITYt Autoootlve Mechanic 

NUMBER OF CREDITS 

EQUIPMENT/TASK 1 2 3 A 5 

M60 TANK 

1^ Replace Engine/Transmission * * * * 

2 Replace Fuel Injection Nozzle * t^t a ti^ 

3 Replace Fuel Injection Pump * it ^ it 

4 Replace Fuel Tank ^ if it 
$ Replace Turbocharger * 



Figure 1. An abbreviated Individual Skill History 
for an automotive mechanic • 



that task when the opportunity arises. AUernatively, when a repair task 
needs to be performed quickly, the Individual Skill History serves to identify 
the most experienced personnel. 

Training Requirements Summary 

Jhe Training Requirements Summary (Figure 2) is designed to Indicate to 
unit training managers and trainers what job exposure is needed, what training 
needs to be developed, and which personnel require the training. The column 
JJGer]iam|s lists those personnel who require training to meet requirements of 
their primary MOS^ and paygrade. For the Equipment/Tasks column, the table is 
organized vertically so that the closer an equipment Is placed to the top of 
the listing,-^ the higher the priority that is required for training on this 
equipment and related repair tasks. Based on the Individual Skill History 
information described above, a computer algorithm generates the names and 
Equipment/Tasks training priorities for a given unit. Notice, for example, 

f^5L?!«^-"^^ ^^^^^^i!^ the Training Requirements Summary as needing 
training in replacing the turbocharger of the M60 tank (Task 5), 

The extreme right-hand column of the Training Requirements Summary 

a training reference. The reference number in this column keys the 
*o a page number in an MPS booklet which contains detailed training 
resource Information arranged by task and equipment. Figulre 3 shows page B79 
from this MPS training resource booklet. Here, marks in the boxes show that 
fourteen sources of training materials are available for Task 5, replacing thu 
turbocharger. ^ 

4.^^ Training Requirements Summary is the single most important record for 
the unit-level training manager. It is an objective statement of training 
requirements listed by task, equipment, and priority, for each individual 
according to his performance and experience. The computer will maintain this 
history and update It as each task is performed, training is received, or 
performance is demonstrated. * 

Training QDDortunit.v Utilization 

The Training Opportunity Utilization information provides a measure of the 
extent of on-the-job training that occurs in the maintenance shop. This 
measure is a ratio of the number of avjilable on-the-job training 
opportunities (I.e., the total number" of repair jobs which enter the 
maintenance shop) to the number of utllizld on-the-.iob training opportunities. 

Figure 4 shows the training opportunity utilization for two different 
MOSS, automotive mechanic and armament mechanic. In the automotive mechanic 
shop section, for example, 250 repair jobs were received by the shop and were 

ShAH^i Jli? J#^Jhf^]^^2^ on-the-^ob training purposes. As indicated by the 
shaded area of the column, about 52 jobs (or about 211) of these 260 were 
actual y utilized for on-the-job training. In other words, inexperienced 
mechanics were assigned to work on 21% of incoming jobs -the remaining jobs 
were performed by experienced personnel. 

Across the nine MOSs tracked by MPS, the utilization of training 
opportunities ranges from 11% to 21$. Based on experience with' maintenance 
shop operations and other MPS data, it appears that a training opportunity 
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TRAINING REQUIREMENTS SUMMARY 



NAMES /MOS 



EQUIPMENT /TASKS 
M60 TANK 



TRAINING 
REFERENCE 



Mechanic 1 (63H) 
Mechanic 2 (63H) 
Mechanic 3 (63H) 



5 Replace Turbocharger 



B79 



Mechanic 2 (63H) 



1 Replace Engine /Transmisjs ion 



B79. 



Mechanic 3 (63H) 



3 Replace Fuel Injection Pump 



B79 



Figure 2. An abbreviated Trianing Requirements Supiary 
for automotive mechanics. 
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utilization rate of about 20% is as high as can be expected given the number 
of personnel available to the unit and the unit's mission requirement. This 
finding suggests that increased attention be given to off-line, special 
training programs: given the constraints on a unit's resources, on-the-job 
training by itself cannot produce mechanics who are we 11 -practiced in all of 
their MOS technical tasks, 

On*the->3ob Training Assignme nt 

Figure 6 indicates the extent to which training managers effectively 
distribute job assignments among inexperienced personnel. It is here in the 
assignment of personnel to jobs that training managers can have the most 
effect on technical skill training. In the automotive mechanic shop, for 
example, about 70% of the total number of inexperienced mechanics (n=31} 
received some type of on-the-job training over the course of eight months. 

Across all the MOSs monitored by MPS, the percent of inexperienced 
mechanics receiving on-the-job training ranges from 30% to 100%, i,e,, there 
is about a three-fold difference in the effective use of incoming jobs for the 
training of inexperienced personnel. This finding clearly indicates that the 
management of on-the-job training can be improved through more effective job 
assignments. Although the opportunities for the conduct of on-the-job 
training may be constrained by factors beyond the control of unit level 
managers, MPS reports can help training managers use these opportunities more 
effectively. 

Frequency of Job/Task Occurrence 

The frequency of occurrence for automotive repair tasks for the M60 tank 
is given in Figure 6. The data reveal that some critical tasks (e,g., 
replacement of the fuel tank and turbocharger) do not occur often in the 
normal garrison environment and hence there is little opportunity for on-the- 
job training on these tasks. MPS data reveal similar results in most other 
MOSs. That is to say,' there are a small number of repair tasks that occur 
very frequently and a larger number of tasks which occur infrequently or not 



The job frequency data raise several Interesting questions. If some 
repair jobs do not occur in the garrison environment, how will repairmen be 
trained to perform the tasks? Should the institutional-setting Advanced 
Individual Training (AIT) courses focus on the infrequently performed tasks or 
ones that are most likely to be encountered in the field? How critical are 
the repair tasks which occur infrequently, and are they likely to be repairs 
which are found in combat situations? ARI is now examining the MPS job 
frequency data in order to determine if there are more effective ways to 
partition training between school and unit environments. 
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Figure 5. On-the-job training assignments for two MOSs. 
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Figure 6. The frequency of automotive mechanic repair 

tasks for the M60 tank. v 
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Mean Repair Time 



Mean repair times (in days) for equipment serviced by automotive mechanics 
are presented in Figure 7. These repair times can be further broken down into 
repair times for specific tasks on each piece of equipment. 

Repair times such as these have utility in the estimation of performance 
standards, system personnel requirements, logistical support t^equirements, and 
equipment design. Typically, the data required for these types^- of estimations 
are expensive and time-consuming. MPS, however, captures the data on a 
longitudinal basis in a simple and routine way. 

National Training Center Data 

Because one company being monitored by MPS participated in the National 
Training Center (NTC) exercises, it 1s possible to compare MPS data from the 
garrison environment with an environment more similar to that of a combat 
situation, e.g., the National Training Center. A sample of findings appears 
in Figures 8 and 9, which show mean repair time and mean team size, 
respectively. 

Mean repair times (in days) across all equipment for two MOSs are shown in 
Figure 8 for both locales. The data suggest that repair times are 
consistently shorter at NTC than in garrison. This difference in repair time 
may reflect a difference between normal garrison operations and the NTC combat 
mode. Hence the team sizes at each locale were analyzed and are shown in 
Figure 9. Here, the results indicate that team size was smaller at NTC than 
in garrison. Therefore, team size (and by implication training mode) is. 
identified as a factor affecting repair time. Comparisons such as these hold 
the potential for generating more accurate standards of repair time in combat, 
as well as generating estimates of unit level trairiilng costs within the 
garrison environment. 

Data from NTC are also useful in another way. ARI is currently looking 
for differences between the kinds of repair tasks which occur in garrison and 
those which occur in combat situations, It is possible that some critical 
combat-related repair tasks do not get adequate on-the-job training in the 
garrison environment; the MPS data is being analyzed to determira whether this 
is so. 



Summary and Discussion 

The goal of MPS is to improve the delivery and conduct of technical 
training by providing relevant information to unit level training managers. 
The data shown above clearly demonstrate the usefulness and value of MPS in 
meeting this goal. Moreover, MPS is a milestone in that It represents the 
first operational military reporting system which provides Information about 
the status of personnel rather than equipment. Reporting systems similar 1n 
scope and intent are now under development by the Navy and Air Force (these 
systems are, respectively, the Enlisted Personnel Individualized Career System 
and the Integrated Training System). 
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Figure 7. Mean repair time (in 'lays) grouped by type 
of equipment for automotive mechanics. 
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Figure 8. Mean repair time (in days) at National Training 
Center (NTC) and in garrison for two MOSs. 



24 



u 



TEAMS IZE 
MEAN 









AAA** 






Axmametit 



***** 



I NTC ^ 



***** 
***** 
***** 
***** 



***** 
***** 
***** 
***** 
***** 



Armaffl en t Au t omo t Iv e 
j Garrison -| 



Figure 9. Mean team size at National Training Center (NTC) 
and in garrison for two MOSs. 
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One of the most important outcomes sought for MPS is to help break the 
pattern of ea'^ly incorrect task performance by technicians because of lack of 
supervision and feedback on performance. MPS attempts to do this by 
identifying training requirements, highlighting early performance trials, and 
i^equiring overt task evaluation by supervisors. As MPS continues to provide 
high quality longitudinal performance data, ARI will seek to determine whether 
this is happening, 

^\t present, ARI has found that the cost of operating MPS is acceptable to 
the units and that the MPS outputs are judged to be of considerable value. 
The most significant use of MPS to date has been to guide job assignments, 
especially for high density MOSs. With the information MPS provides, 
supervisors can readily identify who needs specific kinds of job experience; 
so that when mission requirements permit, proper use of the job environment 
for training and skill development can be achieved. 

On a larger scale, MPS is tegarded as the initial component in a total 
training system for the future. Current efforts are being directed toward 
extending MPS into computer-based training delivery. Based on MPS performance 
'data, those tasks which need to be trained and matched better with the most 
appropriate training delivery system can be targeted, AFi has already 
developed several prototype computer-bas^d maintenance trainers (see Johnson, 
Entwistle, and Gaddis, 1982; Johnson 'and Rouse, 1982) and Is exploring the*- 
^feasibility of a computer link between these devices and the MPS system. The 
promise of such a computer-based training management and delivery system is a 
reduction in both the training cost and training time. 
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